Skip to content

Conversation

@MoLow
Copy link
Member

@MoLow MoLow commented Jul 10, 2022

No description provided.

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Jul 10, 2022
@MoLow
Copy link
Member Author

MoLow commented Jul 10, 2022

CC @aduh95

@MoLow MoLow force-pushed the test-runner-fix-it-concurrency branch from 238a705 to ba13d84 Compare July 10, 2022 14:10
@aduh95
Copy link
Contributor

aduh95 commented Jul 10, 2022

/cc @nodejs/test_runner

@aduh95 aduh95 added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. labels Jul 10, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 10, 2022
@nodejs-github-bot
Copy link
Collaborator

@aduh95
Copy link
Contributor

aduh95 commented Jul 10, 2022

Hum it looks like it makes message/test_runner_desctibe_it flaky. @MoLow do you know where this might be coming from?

=== release test_runner_desctibe_it.js ===
Path: message/test_runner_desctibe_it.js
Error: --- stderr ---
node:internal/test_runner/harness:29
      throw err;
      ^
TypeError: Cannot mix BigInt and other types, use explicit conversions
    at ItTest.report (node:internal/test_runner/test:425:42)
    at ItTest.finalize (node:internal/test_runner/test:418:10)
    at Test.processReadySubtestRange (node:internal/test_runner/test:195:15)
    at ItTest.postRun (node:internal/test_runner/test:388:19)
    at Test.postRun (node:internal/test_runner/test:370:17)
    at process.exitHandler (node:internal/test_runner/harness:79:10)
    at process.emit (node:events:513:28)
Node.js v19.0.0-pre

@aduh95 aduh95 removed the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jul 10, 2022
@MoLow
Copy link
Member Author

MoLow commented Jul 10, 2022

Hum it looks like it makes message/test_runner_desctibe_it flaky. @MoLow do you know where this might be coming from?

@aduh95
yes, it happens when a test is canceled and was not yet assigned a start time. you can see I already fixed that here
adding concurrency introduced a timing condition that made it more likely to happen since previously when a suite ran serially - either a child test did not run or it did, it was never queued.

I suggest waiting for the fix to land, then rebase and make sure test is stable in this branch

@aduh95 aduh95 added the blocked PRs that are blocked by other issues or PRs. label Jul 10, 2022
@MoLow MoLow force-pushed the test-runner-fix-it-concurrency branch from 2964c88 to 8d66424 Compare July 12, 2022 14:49
@aduh95 aduh95 added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. request-ci Add this label to start a Jenkins CI on a PR. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. and removed blocked PRs that are blocked by other issues or PRs. labels Jul 12, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 12, 2022
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@juliangruber
Copy link
Member

I'm surprised by this, is it actually expected to run it tests in parallel?

@ljharb
Copy link
Member

ljharb commented Jul 12, 2022

I don't think it is - jest runs files in parallel (as does mocha, i believe), but i'm not aware of anything that runs individual its in parallel.

@MoLow
Copy link
Member Author

MoLow commented Jul 13, 2022

perhaps this should be a feature that is off by default?

@MoLow
Copy link
Member Author

MoLow commented Jul 14, 2022

Since this PR conflicts main and needs to be updated anyway - I will add the conclusion gathered in this discussion and re-request code reviews

@MoLow MoLow force-pushed the test-runner-fix-it-concurrency branch from 8d66424 to 5eccc6b Compare July 14, 2022 21:30
@MoLow
Copy link
Member Author

MoLow commented Jul 14, 2022

so the current implementation in this PR works this way:

  • in the level of a test/it the default concurrency is 1
  • when running --test, the default will be the number of CPUs, see:

// TODO(cjihrig): Use uv_available_parallelism() once it lands.
const rootConcurrency = isTestRunner ? cpus().length : 1;

  • I want to leave support for boolean value for the concurrency setting out of the scope of this PR since the number of CPUs as a default only makes sense to me at the file level since each file spawns its own process. inside a file this isn't necessarily a good default - I will open a separate issue to discuss this

@MoLow MoLow added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 14, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 14, 2022
@nodejs-github-bot
Copy link
Collaborator

@juliangruber
Copy link
Member

so the current implementation in this PR works this way:

  • in the level of a test/it the default concurrency is 1

The code still looks like it runs all tests in parallel with unlimited concurrency, also the testTimeout has been removed

@MoLow
Copy link
Member Author

MoLow commented Jul 15, 2022

The code still looks like it runs all tests in parallel with unlimited concurrency, also the testTimeout has been removed

the default is currently 1, see #43757 (comment)

@MoLow
Copy link
Member Author

MoLow commented Jul 15, 2022

also the testTimeout has been removed

I am not sure if timeout is needed in describe and if it should support an asynchronous function, anyway - I will sort that out in #43554

@benjamingr benjamingr added the commit-queue Add this label to land a pull request using GitHub Actions. label Jul 15, 2022
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jul 15, 2022
@nodejs-github-bot nodejs-github-bot merged commit a3766bc into nodejs:main Jul 15, 2022
@nodejs-github-bot
Copy link
Collaborator

Landed in a3766bc

@MoLow MoLow deleted the test-runner-fix-it-concurrency branch July 15, 2022 15:55
@danielleadams danielleadams mentioned this pull request Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

author ready PRs that have at least one approval, no pending requests for changes, and a CI started. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.